Bisimulation Metrics for Continuous Markov Decision Processes
نویسندگان
چکیده
منابع مشابه
Bisimulation Metrics for Continuous Markov Decision Processes
In recent years, various metrics have been developed for measuring the behavioural similarity of states in probabilistic transition systems [Desharnais et al., Proceedings of CONCUR, (1999), pp. 258-273, van Breugel and Worrell, Proceedings of ICALP, (2001), pp. 421-432]. In the context of finite Markov decision processes, we have built on these metrics to provide a robust quantitative analogue...
متن کاملBisimulation and Logical Preservation for Continuous-Time Markov Decision Processes
This paper introduces strong bisimulation for continuous-timeMarkov decision processes (CTMDPs), a stochastic model which allows for a nondeterministic choice between exponential distributions, and shows that bisimulation preserves the validity of CSL. To that end, we interpret the semantics of CSL—a stochastic variant of CTL for continuous-time Markov chains—on CTMDPs and show its measure-theo...
متن کاملMetrics for Finite Markov Decision Processes
Markov decision processes (MDPs) offer a popular mathematical tool for planning and learning in the presence of uncertainty (Boutilier, Dean, & Hanks 1999). MDPs are a standard formalism for describing multi-stage decision making in probabilistic environments. The objective of the decision making is to maximize a cumulative measure of longterm performance, called the return. Dynamic programming...
متن کاملContinuous stochastic logic characterizes bisimulation of continuous-time Markov processes
In a recent paper Baier, Haverkort, Hermanns and Katoen [BHHK00], analyzed a new way of model-checking formulas of a logic for continuoustime processes called Continuous Stochastic Logic (henceforth CSL) – against continuous-time Markov chains – henceforth CTMCs. One of the important results of that paper was the proof that if two CTMCs were bisimilar then they would satisfy exactly the same fo...
متن کاملBisimulation for Markov Decision Processes through Families of Functional Expressions
We transfer a notion of quantitative bisimilarity for labelled Markov processes [1] to Markov decision processes with continuous state spaces. This notion takes the form of a pseudometric on the system states, cast in terms of the equivalence of a family of functional expressions evaluated on those states and interpreted as a real-valued modal logic. Our proof amounts to a slight modification o...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: SIAM Journal on Computing
سال: 2011
ISSN: 0097-5397,1095-7111
DOI: 10.1137/10080484x